Understanding variable importances in forests of randomized trees Supplementary materials

We suppose that we are given a probability space (Ω, E ,P) and consider random variables defined on it taking a finite number of possible values. We use upper case letters to denote such random variables (e.g. X,Y, Z,W . . .) and calligraphic letters (e.g. X ,Y,Z,W . . .) to denote their image sets (of finite cardinality), and lower case letters (e.g. x, y, z, w . . .) to denote one of their possible values. For a (finite) set of (finite) random variables X = {X1, . . . , Xi}, we denote by PX(x) = PX(x1, . . . , xi) the probability P({ω ∈ Ω | ∀` : 1, . . . , i : X`(ω) = x`}), and by X = X1 × · · · × Xi the set of joint configurations of these random variables. Given two sets of random variables, X = {X1, . . . , Xi} and Y = {Y1, . . . , Yj}, we denote by PX|Y (x | y) = PX,Y (x, y)/PY (y) the conditional density of X with respect to Y .1